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Abstract 

A holomorphic discrete series can be realized as the space of the holomorphic sections 
of a homogeneous vector bundle on a bounded symmetric domain. We can embed it into a 
degenerate principal series realized as the space of hyperfunction sections of a vector bundle 
on the Shilov boundary by taking the boundary value. We consider an algebraic version 
of higher cohomological analogue of such an embedding for complex reductive groups and 
classical groups of type A. 1 

§ 0. Introduction 

Let G be a real linear reductive Lie group and let Gc its complexification. We denote by go 
(resp. g) the Lie algebra of G (resp. Gc) and denote by a the complex conjugation on g with 
respect to go- We fix a maximal compact subgroup K of G and denote by 9 the corresponding 
Cartan involution. We denote by t the complexified Lie algebra of K. 

We fix a parabolic subgroup P of G with a ^-stable Levi part M. We denote by N the 
nilradical of P. We denote by p, m, and n the complexified Lie algebras of P, M, and N, 
respectively. We denote by Pc, Mc, and Nc the analytic subgroups in Gc corresponding to p, 
m, and n, respectively. 

For X £ m, we define 

<5(X) = itr(ad (X)| n ). 

Then, 5 is a one-dimensional representation of m. We see that 25 lifts to a holomorphic group 
homomorphism '■ Mc — > C x . Defining (,2s\n c trivial, we may extend £25 to Pc- We put 
X = Gq/Pq- Let C be the holomorphic line bundle on X corresponding to the canonical 
divisor. Namely, C is the Gc-homogeneous line bundle on X associated to the character £25 on 
Pc- We denote the restriction of £26 to P by the same letter. 
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For a character rj : P — > C x , we consider the unnormalized parabolic induction "Indp(ry). 
Namely, M lndp(r/) is the A-finite part of the space of the C°°-sections of the G- homogeneous 
line bundle on G/P associated to rj. u \ndp(rj) is a Harish-Chandra (q, A)-module. 

If G/P is orientable, then the trivial G-representation is the unique irreducible quotient of 
"lndp(^25)- If G/P is not orientable, there is a character wonP such that ui is trivial on the 
identical component of P and the trivial G-representation is the unique irreducible quotient of 
u lnd£(£ 25 ®w). 

Let O be an open G-orbit on A. We put the following assumption: 
Assumption A There is a ^-stable parabolic subalgebra q of g such that q G O. 
Under the above assumption, q has a Levi decomposition q = [ + u such that [ is a 9 and 
cx-stable Levi part. In fact [ is unique, since we have [ = <r(q) n q. 
For each open G-orbit O on I, we put 

A = R dimun \0,C) K . &nite . 

From [Wong 1993], Ao is a derived functor module. Namely, in the terminology in [Vogan- 
Zuckerman 1984], we have Aq = A q = A q (0). 

If G is of the Hermitian type and P is a Siegel parabolic subgroup of G, there are two open 
G-orbits in A isomorphic to G/K. (the Siegel upper and lower half planes.) For such orbits, 
Ao are representations in holomorphic or anti-holomorphic discrete series. In this case, we can 
embed Ao into n lndp(^2<5) by taking boundary values at the Shilov boundary. 

We consider the other orbits. Then, the corresponding embedding should be a higher coho- 
mological analogy of boundary value maps. Here, we discuss examples. 

Example 1. Let n be an integer such that n ^ 3. Let G be SOq{tiA) and let P be 
its minimal parabolic subgroup. Then X = Gc/Pc has two G-orbits. One is a closed orbit 
Ar = G/P and the other is an open orbit (say O). In this case, Assumption A holds and we 
have O G/{SO{2) x SO (n - 2, 1)). Since O = X - X R , we have the following long exact 
sequence of cohomologies: 

H n ~ 2 (X, C) — > H n ~ 2 ((D, C) — > H^ C ~ 1 (X, C) — > // n ~ 1 (X, C) — > PL n ~ x {0, C). 

Taking account of n— 1 = dim A, we see H n ~ 2 (X, C) = and // n ~ 1 (X, C) = C from the general- 
ized Borel-Weil-Bott theorem [Kostant 1961]. From [Wong 1993], we have H n ~ 2 (0, ^)^. finite = 
Ao and H n ~ l (0,£) = 0. Since Ajj is an n — 1-dimensional sphere, Ar is orientable. Hence 
H 1 ^~ 1 (X, C) is nothing but the space of hyperfunction sections of the principal series representa- 
tion of G with respect to C. (cf. [Kashiwara-Kawai-Kimura 1986]) Hence, taking A-finite parts, 
we have the following short exact sequence. 

Ao u Inc${i 2S ) -» C 0. 

Hence, the structure of u Ind ( p{^2&) is understood geometrically in this case. 

Example 2. Let n ^ 3. Let G = SOo(n,2) and P be a parabolic subgroup with the 
semisimple part of the Levi part is isomorphic to SOq{ti — 1,1). In this case A = Gc/Pc 
has three open G-orbits. Two of them (say 0+ and OJ) are Hermitian symmetric spaces 
(symmetric domains of type IV). The remaining one (say 0o)is non-Stein and isomorphic to 
G/(SO(2) x SO (n - 2, 2)). Let 0+ and 0_ be the closures of + and C_, respectively. In this 
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case, we have {X - 0+) D {X - O-) = O and (X - 0+) U (X - O-) = X - X R . Here, we put 
Xr = G/P. Hence, we have the following Mayer- Vietris exact sequence: 

H 2n ~ 2 (X - 0+, C) © H 2n - 2 (X - H 2n - 2 (O , C) -»• H 2n ~ l (X - X R , C). 

We also have the following exact sequences. 

H 2n ~ 2 {X,£) -»• H 2n ~ 2 {X - 0+,C) -»• H^- l (X,C), 
H 2n - 2 (X,£) H 2n ~ 2 (X -0-,C) -► R^- X {XX)- 

From [Kostant 1961], we have H 2n ~ 2 (X,£) = 0. We can regard 0+ and C_ as closed convex 
sets in an open cell of X. (Namely, we realize + and O- as bounded symmetric domains.) 
Hence, from the edge-of-wedge theorem (see [Kashiwara-Laurent 1983] Theoreme 1.1.2), we have 
H^-\X,C) = H^~\X,C) = 0. Hence, H 2n ~ 2 (X - 0+, C) = H 2n ~ 2 (X - O-, C) = 0. We 
also have the following exact sequence. 

= H 2n ~ 1 (X,C) H 2n ~ 1 (X -X R ,C)^ H 2 x n R (X,C)- 

Hence, we have 

H 2n - 2 (O ,£) ^ H 2n -\X - X R ,C) ^ H 2 x n R (X,C). 

In this case Xr is orientable if and only if n is even. Hence, the local cohomology H x ^(x,Jd) 
is the space of hyperfunction sections of the degenerate principal series of G with respect to C 
(resp. C ® uj) if n is even (resp. odd). Taking K- finite part, we have 

Ao ^ M lnd^(6«5) (if n is even), 
Ao ^ u \ndp(&5 <S> uj) (if n is odd). 

Such an easy construction of an embedding seems to be difficult to obtain in the general cases. 
However, [Matumoto 1988] , [Kashiwara-Schapira 1990], [Sahi-Stein 1992], [Gindikin 1993], etc. 
give some evidences of the existence of the edge-of-wedge type embeddings as above in more 
general settings. In this article, we consider the following problem in the setting of Harish- 
Chandra modules: 

Problem B Let G be a real linear reductive Lie group and let P be its parabolic subgroup. 
Let O be an open G-orbit in X = Gc/Pc satisfying Assumption A. Does there exist a character 
X of P trivial on the identity component such that Ao ^ "' n dp(£;25 <£> x)? 

If the nilradical of P is commutative, the answer is known more or less by [Sahi 1992], 
[Sahi-Stein 1990], [Sahi 1995], and [Zhang 1995]. For instance, we consider a real rank n group 
G = Sp(n, R) and its Siegel parabolic subgroup P. If n is even, then Problem B is affirmative. 
However, if n is odd, Problem B fails except for holomorphic and anti-holomorphic discrete 
series. 

In this article, we first consider the case that G is a complex reductive group. In this case, 
X = Gc/Pc = G/PxG/P has a unique open G-orbit (say Co). Let g = Lie(G) and p = Lie(P). 
We denote by m a Levi part of p. We fix a Cartan subalgebra f) of g such that f) C p. We denote 
by wo (resp. Wp) the longest element of the Weyl group with respect to (g, f)) (resp. (m, f))). Oq 
satisfies the Assumption A if and only if w^Wp is an involution. 

In fact, we have : 
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Theorem C (Theorem 1.2.6) Assume G is a complex reductive group. Ao ^ u lndp(^2<5) 
if and only if wqw p is a Duflo involution in the Weyl group for (g, h). 

The condition " WQWp is a Duflo involution." is same as the existence condition for a non- 
trivial homomorphism between certain generalized Verma modules ([Matumoto 1993]). In fact, 
Ao can be identified with some non-class one degenerate principal series representation ([En- 
right 1979] and [Vogan-Zuckerman 1984]). In viewpoint of this identification, the embedding in 
Theorem C is nothing but an intertwining operator between degenerate principal seiries induced 
from the homomorphism between generalized Verma modules, (cf. [Collingwood-Shelton 1990]) 

For a Weyl group of type A, any involution is a Duflo involution. We get an affirmative 
answer on the case of G = GL(n, C). Moreover, we have: 

Theorem D (Theorem 3.2.1) Under Assumption A, we have an affirmative answer on the 
case ofG= GL(n,R) and G = GL(n,M). 

In this case, as a submodule of "Indp^^j), Ao is the image of intertwining operator from 
the degenerate principal series representation of which Ao is realized as a quotient. This 
intertwing operator is induced from a homomorphism between generalized Verma modules. 

Finally, we mention the case of G = U(m, n). 

Let G = U(m,n) and let P be an arbitrary prarabolic subgroup of G. In this case, As- 
sumption A automatically holds for all the open orbits. However, there is an obvious necessary 
condition. Since u lndp(^2<5) has a perfect pairing with an irreducible generalized Verma module, 
any submodule of "lndp(^2<5) has the same Gelfand-Kirillov dimension as "Indp^^)- We call 
an open G-orbit O in X = Gc/Pc good, if the Gelfand-Kirillov dimension of Aq equals that 
of u lndp(£2<s)- (We can determine good orbits applying a combinatorial algorithm described in 
[Trapa 2001].) We denote by Q the set of the good open G-orbits on X. We have: 

Theorem E (Corollary 2.10.3) 

SocleHndp^a)) = Ao- 

Here, Socle( lt lndp(^25)) is the maximal semisimple submodule of u lndp(^25)- In this case, 
Socle( u lnd£(f 25 )) is the image of an intertwining operator induced from a homomorphism be- 
tween generalized Verma modules. 

I thank Hiroyuki Ochiai for his informing me of typographical mistakes in earlier version. 

§ 1. General connected complex reductive groups and GL(n, C) 
1.1 Notations 

Let G be a connected complex reductive linear Lie group. Let be the Lie algebra of G, U(q) 
the universal enveloping algebra of g, f) a Cartan subalgebra of g, and A the root system with 
respect to (g, f)). Let W be the Weyl group of the pair (g, h) and denote by wq the longest 
element of W. We fix a positive system A + of A and denote by n the corresponding basis of A. 
We fix a triangular decomposition g = n © f) © n such that n = Yl a eA+ 0" an< ^ ^ = S-«eA+ 0"- 
Here, g a is the root space with respect to a. We consider a Borel subalgebra b = I) + n. 

We denote by go the normal real form of g which is compatible with the above decomposition 
and denote by X ~> X the complex conjugation with respect to go- Then there is an anti- 
automorphism, so-called Chevalley anti-automorphism, X * X of g which satisfies the following 

(l)-(3). 
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(1) *0O = 0O- 

(2) *n = n, *ix = n. 

(3) *X = X (X G fj). 

We extend X ~» *X to an anti-automorphism of t/(fl). 

We define a homomorphism of a real Lie algebra g^gxgbyJ-^ (X, X) for X G g. Then 
the image of this homomorphism is a real form of g x g. Hence, we can regard g x g as the 
complexification g c of g. t c = {(X, — t X) \ X G 5} is identified with the complexification of a 
compact form of g. t c is also identified with g by an isomorphism X ~> (X, — f X) as complex 
Lie algebras. We may regard G x G as a complexification Gc of G. Let F be a standard 
parabolic subgroup of G; namely the Lie algebra p of P contains b. We denote by M (resp. m) 
the Levi part of P (resp. p) stable under the Chevalley anti-automorphism. We denote by n p 
the nilradical of p. Let S be the subset of II corresponding to p; namely S is the basis of the 
root system with respect to (m, h). Put S' = {— woa\a G S}. We denote by p' the standard 
parabolic subalgebra corresponding to S'. We denote by w p the longest element of the Weyl 
group with respect to (m, h). 

1.2 Formulation of the problem and results 

Under the above identification Gc = G x G, the complexification Pc of P is identified with a 
subgroup P x P of G x G. The complex generalized flag variety X = G<c/Pc is identified with 
G/PxG/P. X can be regarded as the set of parabolic subalgebra of which is Ad(Gc)-conjugate 
to Lie(P c ) = p c = p x p. 
We easily have: 

Proposition 1.2.1. X has a unique G-orbit (say Oq). The following five conditions are equiv- 
alent to each other. 

(1) Co contains a 9 -stable parabolic subalgebra of gc . 

(2) p and the parabolic subalgebra opposite to p are G-conjugate. 

(3) p = p'. 

(4) S is stable under the action of —wq- 

(5) WQWp = WpWQ. (Namely woWp is an involution.) 

For X <E m, we define S P (X) = ^tr (ad (X)|„ p ) . 

Then, 5p is a one-dimensional representation of m. We see that 2n<5 p lifts to a holomorphic 
group homomorphism ^2n<5 p : M — > C x for all n G Z. Identifying Mq = M x M, for n,m G 
2Z we can regard Xn,m = £n<5 p ^ Cmi p as a rea l analytic group homomorphism M — ► C x . 
Taking account of the natural projection P — > M, we regard Xn,m as a real analytic one- 
dimensional representation of P. So, we can consider an unnormalized degenerate principal 
series representation: 

u lnd£( X „, m ) = {/ G C°°(G)\f(gp) = XnAP^Ua) (<? G G,p G P)}^ finite . 

If p = b, then <5(, = p = ^ SaGA+ a - F° r * e ^> we define an unnormalized generalized Verma 
module: 

u M p (i) = C/( fl )® f7(rt C t(5p . 

Here, C^ p means the one-dimensional module of p such that m acts on it by tSp. If p = b, 
we simply write u M(t) = u M(,(t). We denote by I p (t) the annihilator of Mp{t) in U(g). From 
[Vogan 1984], M p (t) is irreducible for t ^ — 1. Therefore J p (t) is a primitive ideal for t ^ — 1. 
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Under the identification g c = g x g, we identify U(q c ) = U(g) <8> U(g). We denote by 
u ~^ u (u <E U(g) ) the anti-automorphism of U(g) generated by X ~> —X (X G fl). If V is a 
{/(g)-bimodule, we can regard V as a C/(g c )-module as follows. 

(ui <8> U2)w = t uivv,2 (ui,U2 G U(g),v G V ). 

In particular, we may regard U(g)/Ip(t) as a L r (g c )-module. 
We quote: 

Theorem 1.2.2. ([Conze-Berline, Duflo 1911} 2.12 and 6.3) 

If n is an even integer such that n ^ 2, i/ten "lndp(xn,n) is isomorphic to U(g)/Ip(—n) as 
U (g c ) -modules. 

Together with [Borho-Kraft 1976] 3.6 and the primtivity of Ip(— n) (n ^ 0), we have: 

Corollary 1.2.3. J/n > 2, then u \nd$(x nj n) has a unique irreducible submodule (say Y n ). The 
Gelfand-Kirillov dimension ofY n equals that of u \nd < f(x n ,n) ■ The Gelfand-Kirillov dimension of 
u \n& ( f>(xn,n) /Y n is strictly smaller than that ofY n . 

[Borho-Jantzen 1977] 4.10 Corollar (also see [Jantzen 1983] 15.27 Corollar) implies Ip{—n) = 
V(n-2). 

Since u lndp(x2,2) is the induced module associated with the canonical bundle, we only con- 
sider the case of n = 2. For simplicity, we put / = Ip(— 2) = lp>(0). Let M and N be 
[/(g)-modules. Hom<c(M, N) has natural structure of a £/(g)-bimodule. We introduce a U(g c )- 
module structure on Homc(M, N). We denote the £ c -finite part of Hom(M, N) by L(M,N). 
V L( u M(0),V) defines a category equivalence between a category of highest weight mod- 
ules and a category of Harish-Chandra (g c , 6 c )-modules (Bernstein-Gelfand-Joseph-Enright cf. 
[Bernstein-Gelfand 1980], [Joseph 1979]). Via this category equivalence, the 2-sided ideals of 
U{g)/I correspond to the [/(g)-submodules of u M(0)/I u M(0). In particular Y2 corresponds to 
the irreducible highest weight module with the highest weight Tp — p ([Joseph 1979], also see 
[Joseph 1983] p43). Here, r G W is the Duflo involution associated to the primitive ideal I. 
For integral weights p, v G ()*, we denote by V(u,p) the Langlands (Zhelobenko) subquotient of 
u lnd^(^ +p K^ +p ). From [Joseph 1979] 4.5, we have: 

Theorem 1.2.4. The unique irreducible submodule Y2 o/ u lndp(x2,2) is isomophic to V(—Tp, —p) = 
V(w Tw p, p). 

Next, under the equivalent conditions in Proposition 1.2.1, we compare Y2 with Ao = 
H dimn * (O , £) tc _fi nite . First, we remark: 

Theorem 1.2.5. ([Enright 1919], [Vogan-Zuckerman 1984]) 

Assume that p satisfies the equivalent conditions in Proposition 1.2.1. Then we have 

^ - u lnd«(x2,o)= u lnd^(xo, 2 ). 
Since 25 p = p—wow p p, we see Ao is the Langlands subquotient V{— wow p p, —p) oi u \nd ( ^ 3 {^ WoWp p+p ^ 

Since "M p (— 2) is the irreducible highest weight module of the highest weight — 25 v = woWpp— 
p, we have: 
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Theorem 1.2.6. Assume that p satisfies the equivalent conditions in Proposition 1.2.1. Then 
Ao is isomorphic to the unique submodule of u \ndp(x2,2) if and only if wqW p is a Duflo invo- 
lution in W . 

In case that wqw p is a Duflo involution, the embedding of Ao into u lndp(x2,2) can be 
regarded as intertwining operators between degenerate principal series representation: 

u lnd£( X2 ,o) - u lnd^( X2 ,2), 
u lnd£( X o, 2 ) - u lnd£( X 2,2). 

In fact, the following result holds: 



Proposition 1.2.7. ([Matumoto 1993]) 

Let t be a non-negative even integer. Then we have 

u M p (-t - 2) — u M p (t) 

if and only if wqw v is a Duflo involution in W . 

Taking account of the isomorphism u \ndp(x m ,n) — ( u M p (— m)® u Mp{— n))| g n j^ e ) the above 
intertwining operator are induced from the embedding of the generalized Verma module. 

If W is a Weyl group of the type A, each involution in W is a Duflo involution ([Duflo 1977]). 
Hence, we have: 

Corollary 1.2.8. If G = GL(n,C) and p satisfies the equivalent conditions in Proposition 1.1, 
the socle of u lndp(x2,2) "is isomorphic to Ao - 



§ 2. U(m,n) 
2.1 Root systems 

Let m and n be non-negative integers. We fix a maximal compact subgroup K(m, n) of 
G = U(m,n), which is isomorphic to U(m) x U(n). We denote by 9 the Cartan involution 
corresponding to K(m,n). We fix a ^-stable maximally split Cartan subgroup s H(m,n) of 
U(m,n). We denote by s t)(m, n) the complexified Lie algebra of s H(m,n). We remark that 
all the Cartan subgroup of U(m,n) is connected. We denote by g(m,n) (resp. n))the 
complexified Lie algebra of U(m,n) (resp. K(m,n)). So, we have g(m,n) = Ql(m + n, C) and 
t{m,n) ^ 0l(m,C) x 0t(n,C). 

We denote by Kc(m,n) the complexification of K(m,n). So, Kc(m,n) is isomorphic to 
GL(m, C) x GL(n,C). We identify the complexification of U(m, n) with GL(m + n,C). First, 
we consider the root system A(m,n) with respect to A(g(m, n), s f)(m, n)). Then we can choose 
an orthonormal basis ei, e m -\- n of s l)(m,n)* such that 

A(m,n) = {ej - | 1 ^ «, j ^ m + n, i 7^ j}. 

We choose the above e\, e m+n so that 6>(ej) = e m+n _j + i for all 1 ^ i ^ min{m,n} and 
9{ei) = ei for all min{m, n} < i < m + n — min{m, n}. 

We fix a simple system II(m, n) for A(g(m, n), s {)(m, n)); II = {ei — e2, e m+n _i — e m+n }. 
We denote by A + (m, n) the corresponding positive system. 
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2.2 Standard parabolic subgroups 



Fix 9, s H(m,n), etc. as in 2.1. 

We also consider the particular orthonormal basis e±, ...,e m+n of s f)* denned in 2.1. We 
denote by £1, E m+n the basis of s f)(m, n) dual to ei, e m+n . 

Let k = (ki, k s ) be a finite sequence of positive integers such that 



We put k* = ki + ■ ■ ■ + ki for 1 ^ i ^ s and &q = 0. Hence k* = k. Put m' = m — k and 



Then we have 9{Ai) = —A; L for 1 ^ i ^ s. We denote by a K (m,n) the Lie subalgebra of 
*f)(m, n) spanned by {Ai | 1 ^ i ^ s}. 

We define a subset S^/c) of II(m, n) as follows. 



We denote by M K (m,n) (resp. m K (m, n)) the standard Levi subgroup (resp. subalgebra) of 
U(m, n) (resp. g(m, n)) corresponding to S'(k). Namely M K (m,n) is the centralizer of a K (m, n) 
in U(m, n). 

We denote by P K (m, n) a parabolic subgroup of U(m, n) whose ^-stable Levi part is M K (m, n) . 
We choose P K (m,n) so that the roots in A(m,n) whose root spaces are contained in the com- 
plexified Lie algebra of the nilradical of P K (m,n) are all in A + (m, n). We denote by N K (m,n) 
the nilradical of P K (m,n). 

Formally, we denote by U(0, 0) the trivial group {1} and we denote by GL(k, C) a product 
group GL(fci,C) x • • • x GL(fc s ,C). Then, we have 



GL(k, C) and \J(m',n') can be identified with a subgroups of M K (m,n) up to their automor- 
phisms. The Cartan involution 9 induces Cartan involutions on M K (m,n), GL(«, C), \J(m',n') 
and we denote them by the same letter 9. 

We denote by p K (m, n), m K (m,n), and n K (m,n) the complexified Lie algebras of P K (m,n), 
M K (m,n), and N K (m,n), respectively. 

2.3 Parabolic inductions 

For a general reductive Lie group G and its parabolic subgroup P, we define an unnormalized 
induction as follows. We fix a maximal compact subgroup K and the corresponding Cartan 
involution 9 such that P has a ^-stable Levi part L. We denote by q, t, and [ the complexified 
Lie algebras of G, K,and L, respectively. We denote by U the nilradical of P. Let Z be a 
Harish-Chanda (I, L n K)-module and (tt,H) be a Hilbert globalization of Z. u \ndp(Z) is the 
infinite part of 



k = k± + ■ ■ ■ + k s ^ min{m, n}. 



n' = n — k. 
We put 




S(k) = IT - {e k * - e k *+i, e m+n _ k * - e m+n _ fc * + i|l < i ^ s}. 



M K (m,n) GL(/e,C) x U(m',n')- 



{/ G C°°(G) ® I /(^n) = tt^- 1 )/^) (j£G,f 6L,n£ 17)}. 
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Let m and n be non-negative integers and Let k be a positive integer such that k ^ 
min{m, n}. Let fx and v be complex numbers such that We define a one-dimensional 

representation (t?^, C^) of GL(/c,C) as follows. 

rfc, v (g) = det^d^p (g G GL(fe, C)). 

When we identify M^(m, n) with GL(k, C) x U(m — k,n — k), there is ambiguity coming from 
automorphisms of GL(k,C) and ?7(m — k,n — k). We choose the identification so that the 
differential of the restriction of Cj^„ IE] Ctrivial to s H(m, n) is ^ X^=i e « + ^ Sj=i e m+n-j+i G 
s f)(m, n)*. For a Harish-Chandra (fl(m — k,n — k), K{m — k,n — fc))-module Z, we define 

«/ fc [ M ,,](Z) = «lnd^ n) (C^KZ). 

Taking account of "p-shift", we define the normalized parabolic induction by n I k [fi,u](Z) = 
u I k [fi + m '+%' +k ,i> + m '-%' +k ](Z). 

2.4 ^-stable parabolic subalgebras 

In this article, we denote by N the set of the non-negative integers. 

We discuss ^-stable parabolic subalgebras with respect to U(m, n) (cf. [Vogan 1996] Example 
4.5). 

Let £ be a positive integer. Put 

C ii 

Pe(m, n) = < ((mi, me), (ni, ...,n^)) £ N 1 x N f ^ mj = m, ^ rij = n, m^ + rij > for all 1 < j 
I i=i i=i 

We also put 

F(m,n) = [j P e (m,n), 
e>o 

P(O,O)=P O (O,O) = {((0),(0))}. 

If (m,n) G P(m,n) satisfies (m,n) G P^(m,n), we call £ the length of (m,n). For (m,n) G 
P(m, n), we define 

^(m,n) = diag(/ mi , —I m , I mi , — I ni ) 

Here, for a positive integer 1^ means the kxk identity matrix. If k = 0, we regard Iq as nothing. 
Then we can consider the following realization of U(m, n). 

U(m,n) = {g G GL(m + n, C)|'g/( mi n)S = / (m,n)} • 

Let 9 be the Cartan involution given by the conjugation by I(m,n)- I n this realization, we 
denote by q(m,n) the block-upper-triangular parabolic subalgebra of g(m,n) = gl(m + n,C) 
with blocks of sizes p\ + q±, ....,pe + qe along the diagonal. Then, q(m, n) is a (9-stable parabolic 
subalgebra. The corresponding Levi subgroup U(m,n) consists of diagonal blocks. 

U(m,n) = U(mi,ni) x • • • x U(m e ,ne). 

We denote by £((m,n) the complexified Lie algebra of U(m,n). We denote by o(m,n) the 
nilradical of q (m, n) . 

Via the above construction of q(m, n), K(m, n)-conjugate class of ^-stable parabolic subal- 
gebras with respect to U(m, n) is classified by P(m,n). 



2.5 Cohomological inductions 

First, we discuss the case of ^-stable maximal parabolic subalgbras. 

We fix non-negative integers m and n and ((m',p), (n',q)) G ¥2(171, n). Here, m' = m — p 
and n' = n — q. For an integer h, we define a unitary character rj^' 9 : \J(p, q) — > C x by 

= det(g) h (g£XJ(p,q)). 

We denote by C^ q the representation space of rf[[ q . For a Harish-Chandra (Q(m—p,n — q),K(m — 
p,n — g))-module Z, we define 

W*\h\(Z\ - ( u v B(m,n),K{m,n) ym'+qn 1 n 

n — y '^d((m',p),(n',q)),K(m',n')xK{p,q)J ^ ^ ^h+m+n' ^ ^p+q' 

Here, ( u ^;(^?p ) K ( i?J ) ) ^ (m>0x ^ (Pig) ) P +<? is an unnormalized cohomological induction de- 
fined in [Knapp-Vogan 1995] P 676. If (^S^U^Mm'^xK^ is a cohomological 
induction functor defined in [Knapp-Vogan 1995] p328 or [Vogan 1982], then we have 

HF«[h](Z) = (<#o^ 

Let (m, n) = ((mi, m^), (m, n^)) G P^m, n) and h = (hi, ^) G 1r . We define 

Am,n)[h] = R m ^[h t ](R^-^[ht_i](- ■ ■ (R m ^[h 2 ](C^)) •••))• 

^•(m,n)[bJ is nothing but a derived functor module for q(m,n). 
We quote: 

Theorem 2.5.1. ([Vogan 1984], [Vogan 1988], [Trapa 2001] Lemma 3.5) 

(!) *4(m,n) [h] is a derived functor module in the good range for q(m, n) if and only if 
hi ^ h-i + i for all 1 ^ i < £. In this case, »4(m,n) [h] ^ non-zero and irreducible. 

(2) A(jr\ n)[h] ^ a derived functor module in the weakly fair range for q(m, n) if and only 
if ~'~ 

, 7 mi + m + m i+ i + n i+ i . 

hi - h i+ i ^ - (1 ^ 1 < £). 

(3) vA( m ,n)[h] i s a derived functor module in the mediocre range for q(m, n) if and only if 

hi - h i+ i ^ - maxjmi + rij, nij + nj} - ^ (m k + n k ) (1 ^ i < j ^ I). 

i<k<j 

In this case »4(m,n) [h] is zero or irreducible. 

The mediocre condition is weaker than the weakly fair condition. The weakly fair condition 
is weaker than the good condition. The mediocre condition implies the unitarity of »4(m,n) [h] • 
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2.6 Associated varieties 



It is well-known that the GL(m+n, C)-nilpotent orbits in g(m, n) = gl(m+n, C) are parametrized 
by the Young diagrams with m + n boxes. (Cf. [Collingwood-McGovern 1993]) 

Fix a sequence of positive integers c = (ci, eg) such that c\ + ■ ■ ■ + cg = m + n. Let t G &g 
be a permutation such that c T m ^ c r ( 2 ) ^ ••• ^ c r m. Denote by Y(c) the Young diagram 
consisting of I columns such that the length of the z-th column of Y(c) is c T ^y We denote by 
0(c) the nilpotent GL(m + n, C)-orbit in g(m,n) corresponding to Y(c). 

For (m, n) = ((mi, mg), (ni, ...n^)) S P^(m,n), we put m + n = (mi + m, m^ + 
n^). For (m,n) = ((mi, ...,mg),(ni, ...ng)) £ ¥g(m,n), the Richardson orbit with respect to 
q(m, n) coincides with 0(m + n). (Kraft, Ozeki, Wakimoto, see [Collingwood-McGovern 1993] 
Theorem 7.2.3 ) Namely, 0(m + n) is a unique nilpotent GL(m + n, C)-orbit in g(m, n) such 
that C(m,n) n p(m, n) is open in t>(m,n). (Here, t)(m,n) is the niradical of q(m, n.) It is 
known that dim O (m + n) = 2 dim x> (m, n) . 

Let fl(m, n) = t(m,n) @ s(m,n) be the complexified Cartan decomposition with respect to 

6. 

The nilpotent Kc(m, n)-orbits in s(m, n) are parametrized by (the equivalence classes of) the 
signed Young diagrams of signature (m, n). (See [Collingwood-McGovern 1993]) Here, a signed 
Young diagram is a Young diagram in which every box is labeled with + or — sign in such a 
way that signs alternate across rows. We call two signed Young diagrams equivalent, if one can 
be obtained from the other by interchanging rows of equal length. The signature of a signed 
Young diagram is the ordered pair (m, n), where m (resp. n) is the number of boxes labeled + 
(resp. — ). For a signed Young diagram T, we denote by |T| the Young diagram obtained by 
earasing all + and — on Y. We call \T\ the shape of T. For a Young diagram consisting of m + n 
boxes, we denote by § mjn (Y) the set of the equivalence classes of the signed Young diagrams of 
signature (m, n) whose shape is Y. 

The above parametrization of the nilpotent Kc(m, n)-orbits in s(m, n) is compatible with the 
parametrization of GL(m + n, C)-nilpotent orbits in g(m, n) = Q\(m + n, C) via Young diagrams. 
Namely, for any nilpotent Kc(m, n)-orbit O in s(m, n) corresponding to a signed Young diagram 
T, the nilpotent GL(n + m, C)-orbit Ad(GL(m + n,C))0 corresponds to \T\. For a nilpotent 
GL(m + n, C)-orbit O in fl(m, n) corresponding to a Young diagram Y, On s(m, n) is the union 
of the nilpotent Kc(m, n)-orbits corresponding to the elements of S m>n (Y). 

For a Harish-Chandra (fl(m, n), K(m, n))-module M, we denote by Dim(M) (resp. Ass(M)) 
the Gelfand-Krillov dimension (resp. the associated variety) of M. (cf. [Vogan 1978], [Vogan 
1991]) For an irreducible Harish-Chandra (fl(m, n), K(m, re))-module M with an integral in- 
finitesimal character, Ass(M) is the closure of a single nilpotent Kc(m, n)-orbit in s(m, n). 
([Barbasch- Vogan 1982]) 

Lemma 2.6.1. ([Trapa 2001] Lemma 5.6) 

Let Z be a Harish-Chandra (g(m, n), K(m,n)) -module such that Ass(Z) is the closure of 
a single nilpotent Kc(m,ri)-orbit in s(m, n) corresponding to a signed Young diagram T. Let 
h be a sufficiently large integer so that the ccohomological induction R p,q [h](Z) is in the good 
range. Then Ass(R p,q [h](Z)) is the closure of a nilpotent Kc(m + p,n + q)-orbit (say O) in 
s(m + p,n + q). Then, the signed Young diagram corresponding to O is obtained by adding p 
pluses and q minuses, from top to bottom, to the row-ends of T so that 

(1) at most one sign is added to each row-end, 
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(2) the signs of the resulting diagram must alternate across rows, and 

(3) add signs to the highest row possible. 

( The resulting diagram may not have rows of decreasing length. If necessary, we can choose 
another signed Young diagram equivalent to T in order to obtain a signed Young diagram as the 
result of the above procedure.) 

Remark As explained in [Trapa 2001], we can calculate the associated varieties of derived 
functor modules of U(m,n) using Lemma 2.6.1. 

For (m,n) G P(m,n), Dim(^ m>n [h]) ^ ^ dimO(m, n). We call (m,n) G P(m,n) normal, 
if Dim(^4m,n[h]) = ^dimO(m, n) holds in the good range. It is kown that for a normal 
(m, n) G P^(m,n) and any mediocre h G Z , we have Dim(^m,n[h]) = ^dimO(m, n) and 
Ass(>4m,n[h]) = Ass^m.njh']), where h' is any parameter in the good range, (cf. [Trapa 
2001]) Fix a sequence of positive integers c = (ci, q) such that c\ H + q = m + n. Put 

O(c) = {(m, n) G P^(m, n) | (m, n) is normal and mi + rij = q for all 1 ^ i ^ ^} 

Examining the procedure described in Lemma 2.6.1, we can easily have: 

Corollary 2.6.2. Let c be a sequence of positive integers as above. The correspondence (m, n) ~^ 
-*4ss(*/4(m,n)[h]) induces a bisection 

$c : O(c) ^ S m ,„(y(c)). 

2.7 The annihilator of a degenerate principal series representation 

Let m and n be non-negative integers. Let k = (fei, fc s ) be a finite sequence of positive integers 
such that k = k\ + ■ ■ ■ + k s ^ min{m, n}. Let u = (u±, ...,u s ) and v = (vi, f s ) be sequences of 
complex numbers such that Uj + Vi G Z for all 1 < i ^ s. Let fteZ. We consider a degenerate 
principal series representation of U(m, n) as follows. 

n C,n[u; /»; v] = n i k ^iM{ n i k2 \^M{- ■ ■ r/ fes K,^](cr fc ' n " fe ) ■■■))■ 

If m = n = /c, we regard C^'° as the trivial representation of the trivial group. In this case 
n I^ in \u\ h; v] does not depend on h. We denote by 5^ an element of C s whose i-th entry is 

2 , where fe* = fci + • • • + /jj. We denote by J(u, ft,, v) the annihilator of n I^ in \uj /i; v] in 

the universal enveloping algebra U(g(m, n)). 

We define a normalized generalized Verma module as follows. Let p be a parabolic subalgebra 
°f = 0(m + n). Let p = I + n be a Levi decomposition of p. We define a one-dimensional 
representation — p p of 1 by — pp(X) = ^(ad(A)|„) (X G [). For a one-dimensional representation 
£ of [ we extend £ ® — p p to a one-dimensional representation of p as usual and define n M p (£) = 

u(e) ®u( P ) 

We define a weight on s f}(?rt, n) by 

s I ki \ m+ri— 2fc s / fci 

em+n-fc+fcj'_ 1 +j 

i=l \j=l / j=l i=l \j=l 

Obviously, £(u, /i,v) can be extend to a one-dimensional representation of I. We denote by 
Pk(^jI') the opposite parabolic subalgebra to p K (m, n). n i^ n [u ; ft;v] and n Mp K (rn,n) 
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have a perfect pairing. ra Mp K ( m n )(^(u, h, v)) and n -Mp / mjJl )(— £(u, h, v)) also have a perfect pair- 
ing. 

Hence we have: 

Lemma 2.7.1. J(u,h,v) coincides with the annihilator o/ n Mp K ( m j7l )(£(u, fo, v)). 

Let c = (ci, q) be a sequence of non-negative integers such that c± + ■ ■ ■ + q = m + n. 
Put c* = ci + • • • + Cj and Cq = 0. We define a subset 5[c] of II(m, n) as follows. 

5[c] =n-{e c * -e c?+ i|l 

We denote by p(c) the standard parabolic subalgebra of Q(m, n) associated to 6*[c]. Under 
appropriate realization of 0(m, n) as Ql(m + n, C), p(c) is the block-upper-triangular parabolic 
subalgebra of Ql(m + n, C) with blocks of sizes c\, ...,cg along the diagonal. If q = for some 
i, "a block of size 0" means nothing. Namely, we simply neglect it. We denote by p(c) the 
parabolic subalgebra opposite to p(c). For a sequence h = (hi, hg) of complex numbers, we 
define a weight on s \)(m,n) as follows. 

t / c, 

i=l \j=l 

Hereafter, we consider a degenerate principal series representation n I£ n n [u; h; v] with an 
integral infinitesimal character. Namely, we assume that u — 5£ in >Y + 5^ n G 7L S . We define 
h = (hi, /i2s+i) as follows. 

hi = h, 

h 2 i = u s - i+ i (l^i^s), 
h-2i+i = Vi (1 ^ i ^ s). 

We also define a sequence of positive integer c by 

ci = m + n — 2k, 

C2i = C 2 i+l = k s - i+ i (1 ^ i ^ s). 

We define 

S(u, /i, v) = {r G e 2s +i | /i r( i) ^ • • • ^ /i T (2s+i)}- 

Obviously S(u, /i,v) is non-empty. For r G ©2s+i> put c r = (c T (i), c T ( 2s+1 )) and h r = 
(/i r (i), ^ T (2s+i)) F rom [Borho-Jantzen 1977] 4.10 Corollar and Lemma 2.7.1, we have: 

Proposition 2.7.2. 7/r G S(u, h, v), i/ten J(u, /i, v) coincides with the annihilator o/ n Mp( C T)(£(h' 

From [Vogan 1984], we see that ™Mp( C T)(£(h' r )) is irreducible for r G S(u, /i, v). Hence we 
see that J(u, h, v) is a primitive ideal. 
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2.8 The associated variety of a degenerate principal series representation 

For a Harish-Chandra (g(m, n), K(m, n))-module Z, we denote by [Z] the distribution charac- 
ter of Z. Let V and M be Harish-Chandra (fl(m, n), K(m, n))-modules. We write V ~ M, 
if a virtual character [V] — [M] is a linear combination of the distribution characters of ir- 
reducible Harish-Chandra (fl(m, n), K(m, n))-modules whose Gelfand-Kirillov dimensions are 
strictly smaller than Dim(V). 

As in 2.7, we consider a degenerate principal series representation n /^ n [u; h; v] with an 
integral infinitesimal character. We also define c and h from (u, h,v) as in 2.7. We have: 

Proposition 2.8.1. Assume that S(u, /i,v) contains the identity element e. Then, we have 

(m,n)eO(c) 

Fere, h = (hi,..., h 2s +i) is defined by hi = hi — ™±rp£ + C *_ 1 (1 ^ i ^ 2s + 1). 

Proof. If /i2i = /i2i+i for 1 ^ z ^ s, the irreducible decomposition of n /^ n [u;li;v] is given in 
[Matumoto 1996] Theorem 3.3.1. Each irreducible component is a derived functor module. So, 
the statement of the proposition in this particular case is obtained by examining the method of 
computation, via Lemma 2.6.1, of the associated varieties of derived functor modules appearing 
the decomposition formula. The general case is obtained by applying the standard argument of 
the translation principle into the weakly fair range. ([Vogan 1984], [Vogan 1988], [Vogan 1990]) 
■ 

Remark We may prove Proposition 2.8.1 directly using [Matumoto 2002] Theorem 2.2.3. 
Remark that 0(c) is the Richardson orbit with respect to p K (m,n). We denote by 0(c) its 
closure. 

It is easy to see that Ass( ra /^ n [u; h; yj) does not depend on the choice of the parameter 
(u; h;v) such that n /^ n [u; h;v\) has an integral infinitesimal character. Hence, we have: 

Corollary 2.8.2. For any n ^m„[u;/i;y] with an integral infinitesimal character, we have 

Ass( n I^ n [u; h-y]) = 0{^j n s(m,n) 

From [Schmid-Vilonen 2000], we can identify the wave front sets of derived functor modules. 
The wave front set of the degenerate principal series represntation is obtained by [Barbasch- 
Vogan 1980]. Hence, Corollary 2.8.2 is equivalent to the following result. 

Corollary 2.8.3. The real Lie algebra Lie(N K (m,n)) intersects all the real form of the Richard- 
son GL(m + n,C) -orbit in gl(m + n, C) with respect to p K (m,n). 

Remark The corresponding statement to Corollary 2.8.3 is incorrect for a general semisim- 
ple Lie algebra. For instance, the Jacobi parabolic subalgebra of sp(2, R) gives a counterexample; 
there are three real forms but the nilradical intersects only one of them. 

2.9 Irreducible constituents of the maximal Gelfand-Kirillov dimension 

As in 2.7, we consider a degenerate principal series representation n [u; h; v] with an integral 
infinitesimal character. We also define c and h from (u, h, y) as in 2.7. 
We have: 
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Theorem 2.9.1. For r G E(u, h,v), we have 

n I^ n [mh,v)~ ^ ( rn,n)[h T ] 
(m,n)eO(c T ) 

Here, Yv = {h\, fa^s+i) * s defined by h\ = h T u-\ — m+ " 2 Ct(i) + c T *_ 1 (1 ^ i ^ 2s + 1), where 
c T *_ 1 = c T m + • • • + c T (j_i) /or 2 ^ i ^ 2s + 1 and c t q = 0. 

Proof. Applying the standard argument of the translation principle into the weakly fair range 
([Vogan 1984], [Vogan 1988], [Vogan 1990]), we may assume that h T ^ 3> ■ ■ ■ 3> h T ^ 2s +i)- 
From [Barbasch- Vogan 1983] (also see [Trapa 2001] section 6), we see 

(1) An irreducible Harish-Chandra (g(m, n),K(m, n))-module with an integral infinitesimal 
character is determined by its annihilator and its associated variety. 

(2) The associated variety of an irreducible Harish-Chandra (fl(m, n), K(m, n))-module 
with an integral infinitesimal character is irreducible. 

Let (m, n) G 0(c r ). From [Vogan 1986] Proposition 16.8, we see the annihilator of «4(m,n) [h ] 
contains the annihilator of ra Mp( C T)(£(h r )). Since Dim(^4( m n ) [h. r ]) = ^ dim 0(c), Proposition 
2.7.2 implies that the annihilator of -4( m ,n) [fl] coincides with J(u, h,v). So, taking account of 
the above (1) and (2), we have 

B /£, B |u;&;v]~ © ^n)iT" M . 
(m,n)eO(c T ) 

Here, n(m, n) are positive integers. 
Hence, we have only to show that: 

(O) The number of irreducible constituents of the maximal Gelfand-Kirillov dimension in 
ra I^ ra [u; h; v] equals the number of signed Young diagrams of the shape Y(c). 

Since the datum (u, h,v) is recovered from h, we write /[h] = "/^[ui^v], JQl) = 
J(u, h, v), and S[h] = S(u, /i, v). 

We regard the symmetric group &2s+i as a Coxeter group such that a set of transpositions 
+ 1) | 1 < i < 2s + 1} is the simple reflections. We denote by £(t) the length of r G ©2 S +i- 

We prove (O) by the induction on £(t). The case of £(t) = follows from Proposition 2.8.1. 

So, we assume 1(t) > 0. 

Then there exists some 1 ^ i ^ 2s + 1 such that the length of r' = r o (i, i + 1) is strictly 
smaller than r. Since we assumed that h T ^ S> /i r (j+i) S> /i r (i+2)) there is a positive integer ro 
such that /i r (j+i) S> /i r (j) - ro > /i r (i+2)- F° r r G N, we define h(r) = (h(r)i, h(r)2 S +i) by 
h(r)j = hj for j / r(z) and h(r) T ^ = h T u\ — r. Then, we see 2(h) = {t} and H(h(ro)) = {t'}. 
For r G N, we define weights on s f)(m,n) as follows. 

Cr(j) ( c • - 1 \ 

■4> = 2 ( ^C?) + "^2 d ) &cT U+ d 

l^j42s+l d=l V J 

C r(i) / -1 \ c r(i+l) / -. N 

/ c r(i) ~ 1 \ v^" 1 / C T(i+l) ~ \ 

il>(r) = rf, + ^ I h T(i) - r + d J e cT »_ i+(i + ^ ( ^(i+i) + 7, d I e c r* +d 

d=l ^ ' d=l ^ ' 

C T(i + l) , -i \ C T(i) , -I \ 

// / C T<i+l) ~ i \ I c r(i) ~ \ 

4>{r)=ip+ 2^ I h r(i+i) + 2 d J 6cT r-i +(i + 2^ ( 'MO - r + ^ d J ^ti+c^i+ij+d- 

d=l ^ / d=l ^ / 
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Then, tp(r) and ip'(r) are contained in the same Weyl group orbit and represent the same 
infinitesimal character. In fact, they are the infinitesimal character of I[h(r)]. 

For X, fi G s t)(m,n)* such that A — ji is integral, we denote by the translation functor. 
Namely, if Z is a Harish-Chandra module with a generalized infinitesimal character A, T^(Z) = 
Pfi(Vfj,—\ ®c Z). Here, V^-x is the irreducible finite-dimensional representation with an extreme 
weight [x — A and is the projection functor to the generalized infinitesimal character fj,. 

For non-negative integers r and d such that r > d, we put 

n-r _ rp-4>{r) rp^{r-l) ^(d+l) 
1 d — 1 ^( r -l) ° 1 tp(r-2) ° ' ' ' ° 1 i/>(d) ' 

r ,d _ T V'(d) T ip'(d+1) T V'(r-l) 
1 r - 1 ^'(d+l) ° 1 ^'{d+2) ° • • • ° J ^,'( r ) • 

Using the idea of the proof of Trapa's result([Trapa 2001] Lemma 3.13) together with [Vogan 
1982] Lemma 7.2.9 (b), [Vogan 1988] Lemma 4.8, and Proposition 2.7.2 above, we also have: 

Lemma 2.9.2. 

(1) IfreN satisfies r ^ h r(i) - h r{i+1) + CtW ~ 2 Ct( ' +1) , we have [T r (/[h])] = [/[h(r)]]. 

(2) We fix ro G N as above. If r G N satisfies r ^ h T ^ — /i T (j + i) — 



g T(i)- C T(i + l) 



, we have 



[T-(/[h(r )])] = [/[h(r)]]. 

We denote by TC(r) the set of isomorphism classes of irreducible Harish-Chandra modules Z 
such that J(h(r)) annihilates Z and Dim(Z) = ^ dim 0(c). If we apply [Vogan 1990] Corollary 
7.14 to our setting, we have: 



C T(i)- C T(i + l) 

2 



. (We easily see such 



Lemma 2.9.3. We fix r G N such that \r — h T ^ + /i r (j + i) | ^ 

an r exists.) 

(1) Tq gives a bijection ofTt(0) onto Tt(r). 

(2) For sufficiently large r$ G N, T'^ Q gives a bijection ofH(r$) onto Tt(r). 

From Lemma 2.9.2, Lemma 2.9.3, and the exactness of the translation functors, we see that 
the number of irreducible constituent of the maximal Gelfand-Kirillov dimension in I[h\ equals 
the number of those in i[h(ro)]. From the assumption of the induction, (O) holds for 7[h(ro)]. 
Therefore, we have (O) for I[h]. Q.E.D. 



2.10 The socle of a degenerate principal series representation with a finite- 
dimensional quotient 

In order to obtain the main result, we prepare the following lemma in the general setting. 

Lemma 2.10.1. Let G be a real reductive group and let K be a maximal compact subgroup of 
G. We denote by q the complexified Lie algebra of G. For a Harish-Chandra (q, K)-module M, 
we denote by M h the Hermitian dual of M. (cf. [Vogan 1984]) 

Let I be a Harish-Chandra (g, K)-module and let \E' : I h — > / be a (g, K)-homomorphism. 

We assume the following three conditions (al)-(a3). 

(al) Dim(Kernel(^)) < Dim{I h ). 

(a2) Let V be an arbitrary irreducible constituent of I such that Dim(V) = Dim(I). Then, 
V h = V and the multiplicity ofV in I is one. 

(a3) Let W be any submodule of I, then Dim(W) = Dim(I). 
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Then, we have 

(1) Socle(I) = ^(I h ). 

(2) Dim(I /Sode(I)) < Dim(I). 

Here, Socle(i) is the maximal semisimple submodule of I. 

In particular, any irreducible constituent V of I such that Dim(V) = Dim(I) is a submodule 
of I. 

Proof. Let V be any irreducible submodule of I. Then V h = V has to be realized as a quotient 
of I h . From (a3), we have Dim(V) = Dim(J). From (al), we see ^>(I h ) contains an irreducible 
constituent isomorphic to V. From (a2), we have V C ty(I h ). Hence Socle(J) C ty(I h ). 

In order to show (1), we have only to show that ty(I h ) = I h /Kernel(\l/) is semisimple. 
Let W be any irreducible quotient of I h /Kernel^) . Then, W is also a quotient of I h . So, 
W h is realized as a submodule of I. Hence, Dim(VF^) = Dim(W) = Dim(7) from (a3). So, 
W = W h C Socle(J) C ty(I h ) Hence, I h /Kernel(^) contains a submodule isomorphic to W. 
(a2) implies the multiplicity of W is one. Since any irreducible quotient of I h /Kernel (VP) is a 
submodule, ty(I h ) = I h /Kemel(^>) is semisimple. 

(2) follows from (al) and (1). ■ 

Now, we state the main result. 

Theorem 2.10.2. Assume u = (u±, ...,u s ),v = (v±, v s ) £ 7L S and h € Z satisfy u\ ^ ■ ■ ■ ^ 
u s ^ h ^ v s ^ ■ ■ ■ ^ v\. Put c(m, n, k) = (k\, k s ,m + n — 2k*, fc s , k\) G N 2s+1 . We denote 
by 5^ an element of C s whose the i-th entry is m+n ^ i=l f w here k* = k± + ■ ■ ■ + hi. 

(1) We have 

Socle( n C jn [u + (5^ n ;/i;v-^ ) J) ^ .A(m,n)K, -, ««, h, v s , vi]. 

(m,n)6Q(C(m,n;w)) 

(2) There exists an embedding of a generalized Verma module: 

n M Mmyn) (-t(u + <y« >n , /i, v - ^ "M p4min) (-e(v - <y« >n , /»,u + c >n )). 

T/tis embedding induces an intertwining operator 

& - n I K \v- 5 K ■h-u + 5 K ]) -> n / K [u + <5 K -h-v -5 K ]) 
Moreover, we have Image(^) = Socle( n /^ n [u + /i; v — 5^ n ]). 

Proof. From the standard argument of translation principle, we may assume that u\ = U2 = 
■ ■ ■ = U s = h = V s = ■ ■ ■ = V\ = 0. For simplicity, we put I = n Im,n[^m,n'i ^ ~ ^m,n\) an d 

p = p K (m,n). Then, I h = n I^ n [-5^ n ; 0; +<% j7 J). Moreover, I (resp. I h ) has a perfect 
pairing with a generalized Verma module u M p (— 2) (resp. u M p (0)). (See 1.2.) In this case 
p satisfies the equivalent conditions in Proposition 1.2.1. Hence, Proposition 1.2.7 implies that 
u Mp(— 2) u Mp(0). This induces an intertwining operator ^ : I h ^ I (for instance, see 
[Collingwood-Shelton 1990] 2). Since Kernel(^) has a perfect pairing with u M p (0)/ u M p (-2) 
and Dim( u Mp(0)/ u M p (-2)) < Dim( u M p (-2)) = Dim(J), we have Dim(Kernel(tf)) < Dim(J). 
Since "M p (— 2) is irreducible, any submodule of I also has a parfect pairing with "M p (— 2). This 
implies that any submodule of I has the same Gelfand-Kirillov dimension as /. Together with 
Lemma 2.10.1, we can apply Theorem 2.9.1 to our setting and obtain the result. Q.E.D. 
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The Matsuki duality ([Matsuki 1988]) tells us Kc(m, re)-conjugacy class of ^-stable parabolic 
subalgberas {q(m, n) | m + n = c(m,n, k)} parameterizes the open U(m, n)-orbits of the com- 
plexified generalized flag variety X(m,n, k) = GL(rre + n,C)/P K (m,n)c- We denote C the 
canonical line bundle on X(m,n, n). For an open U(m, n)-orbit O in X(m,n, k), we put 

Ao = HS (£>,£)K(m,n)-hmte = Am,n)[0, 0]. 

Here, S = dim(p(m, n)nf(m, re)) and q(m, n) is the ^-stable parabolic subalgebra corresponding 
to O. We call an open U(rre, n)-orbit O in X(m, n, k) good, if Dim(Ao) = dimX(m, n, k). Then 
we may rewrite Theorem 2.10.2 as follows. 

Corollary 2.10.3. 

Soclerind^)^))^ Ao. 

O : a good open U(m,n)-orbit in X(m,n,n) 

§ 3. GL(n,R) and GL(n,H) 

3.1 Parabolic subalgebras and generalized flag varieties 

Let n be a positive integer greater than 1. Let K be one of M, C, or H. We consider G = GL(n, K). 
We denote by S H the Lie group of re x n diagonal non-singular matrices in G. Fix a maximal 
compact subgroup K of G. We denote by 9 the corresponding Cartan involution. We may 
choose the maximal compact subgroup K appropriately so that s H{n) is ^-stable. We denote 
by (resp. t) the complexified Lie algebra of G (resp. K). We denote by s f) the complexified 
Lie algebra of S H. Then, s t) is a maximally split Cartan subalgebra of g. 

Let c = (ci, c s ) be a sequence of positive integers such that c\ + • ■ ■ + c s = n. We denote 
by -P(c) the block-upper-triangular parabolic subgroup of G with blocks of sizes ci, ...,c s along 
the diagonal. We denote by -Pc(c) (resp. Gq) the complexification of P(c) (resp. G). We denote 
by p(c) the complexified Lie algebra of -P(c). We denote by m(c) the Levi subalgebra of p(c) 
such that s f)(re) C m(c). and denote by M<c(c) the corresponding complexified Levi subgroup. 
We denote by n(c) the nilradical of p(c). 

We denote by wq (resp. wc) the longest element of the Weyl group for (gl(re, C), s f)) (resp. 
(m(c),*h)). 

From [Matsuki 1979], we easily see the full flag variety of Gc has a unique open G-orbit. 
Hence, an arbitrary generalized flag variety X(c) = Gc/Pc(c) has a unique open G-orbit (say 
0c). We regard as X(c) the set of parabolic subalgebras of q which are Ad(Gc)-conjugate to 
p(c). The following result is more or less known. 

Proposition 3.1.1. Let c = (ci, c s ) be a sequence of positive integers such that c± + ■ ■ ■ + c s = 
re. Then, the following conditions are equivalent to each other. 

(1) Oc contains a Q -stable parabolic subalgebra of q . 

(2) p(c) and the parabolic subalgebra opposite to p(c) are G-conjugate. 

(3) For any integer 1 ^ i ^ |, we have c-i = c s -i + \. 

(4) wqWc = wcWq. (Namely, wqWc is an involution.) 
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We denote by wc the canonical line bundle on X(c). Namely, tog. is a homogeneous line 
bundle associate with the character of Mc (c) : 

M c (c) B g det (Ad( 5 )| n( c)) G C x . 

3.2 The socle of degenerate principal series representation with respect to 
canonical bundle 

Assume that c satisfies the equivalent conditions in Proposition 3.1.1 and choose a ^-stable 
parabolic subgroup q(c) in Oc such that q(c) has a Levi part [(c) which is a comlexified Lie 
algebra of its normalizer L(c) in G. We denote by u(c) the nilradical of q(c) 

Put Aoc = ^ 5 (0c,^c)K_ finite = A,(c)(0). Here, S = dim(* n u(c)). 

The case of K = C is discussed in Corollary 1.2.8. For the remaining cases, we have: 

Theorem 3.2.1. 

(1) IfK. = H, then Ao c is a unique irreducible submodule of u \nd ( f^(uJc) ■ 

(2) If K = M, then there is a one- dimensional representation x which is trivial on the 
identical component of P(c) and Ao c is a unique irreducible submodule of u \nd P ^.(uc <8> x)- 

(3) In any case, Ao c is the image of an intertwining operator induced from an embedding 
of a generalized Verma module. 



Proof. We consider the case of X = H. Write EI = C + jC. We put K = Sp(n) = {g G 
GL(fc,H)|*<7<7 = Ik}- Then we regard g[(fc,C) as a real Lie subalgebra of g[(k,W). For £ £ Z and 
t G v^— IK, we define a one-dimensional unitary representation & of GL(k, C) as follows. 



det(g) V 



I det fa) | ,/ 

Let q(k) be a ^-stable parabolic subalgebra with a Levi decomposition q(fc) = Ql(k, C)+u(k). We 
choose the nilradical u(k) so that ^ is in the good range with respect to q(k) for sufficient large 
£. Derived functor modules with respect to q(k) is called quarternionic Speh representations. 

{*) A k{£) ~ ( % (fe)i O(fe) ) '{&+2k) («€/,). 

For £ £ Z, Ak{£) is derived functor module in the good (resp. weakly fair) range in the sense of 
[Vogan 1988] if and only if £ ^ (resp. £^ -k). 

We define d = (di, d^+ij) by dj = 2q = 2c s _j + i (1 ^ i ^ |). Here, [^p] = max{j G 

^ I 3 ^ ^ s is °dd, we put ds+i = Cs+i. We put d* = d± H h dj. The Levi part M(d) 

of P(d) can be identified with GL(d 1 ,M) x* ■ ■ x GL(d [1 +i ] ,M). 

For g 6 GL(fc,EI), we denote by det(g) G M x the noncommutative determinant ([Artin 1957] 
pl51). For t G R, we denote by xt a character of GL(fc,H) defined by Xt(<7) = det(^)*. 

More or less [Vogan 1986] tells us (also see [Matumoto 2002] 2.4 ) 

/[§] 

(*) ^c= n|nd ? ( d) I ®A di (2n-2d*)®id 

Here, id means the identity representation of GL(n — d*s],H). (We regard GL(0,H) as the 
trivial group.) "®"in the above formula (it) means an external tensor product. We quote: 
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Lemma 3.2.2. ([Sahi 1995], [Zhang 1995]) 

A 2k (£) ~ "Ind^f (x/+* ® X-t-k). 

We define c' = (c[, ...,d s ) by = c' 2i = Cj = c s _j + i (1 < % ^ |). If s is odd, we put 

c s = Cs+i. Combining the above (if) and Lemma 3.2.2, we can easily see 

2 

/[§] 

Since M(c) is G-conjugate to M(c'), Harish-Chandra's character theory implies the following 
character identity. 

= Hnd^K)]- 
Hence, 

We can apply an argument similar to the proof of Theorem 2.10.2 and we have the desired result 
for the case of G = GL(n, H). 

For the case of G = GL(n, R), we can prove the result similarly. We may apply [Speh 1983] 
instead of (*). The corresponding result of result of Lemma 3.2.2 is also obtained in [Sahi-Stein 
1990], [Sahi 1995], and [Zhang 1995]. (Also see [Howe-Lee 1999]) ■ 
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